AutoML Leaderboard

Best model	name	model_type	metric_type	metric_value	train_time
	1_Baseline	Baseline	logloss	1.06944	3.24
	2_DecisionTree	Decision Tree	logloss	0.591085	13.58
	3_Default_Xgboost	Xgboost	logloss	0.504764	16.67
	4_Default_NeuralNetwork	Neural Network	logloss	0.565377	4.86
	5_Default_RandomForest	Random Forest	logloss	0.547393	14.93
the best	Ensemble	Ensemble	logloss	0.50403	0.33

AutoML Performance

AutoML Performance Boxplot

Features Importance

Spearman Correlation of Models

Summary of 1_Baseline

<< Go back

Baseline Classifier (Baseline)

• n_jobs: -1
• num_class: 4
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

2.7 seconds

Metric details

	Extreme	Major	Minor	Moderate	accuracy	macro avg	weighted avg	logloss
precision	0	0	0.593171	0	0.593171	0.148293	0.351851	1.06944
recall	0	0	1	0	0.593171	0.25	0.593171	1.06944
f1-score	0	0	0.744642	0	0.593171	0.18616	0.441699	1.06944
support	223	647	2623	929	0.593171	4422	4422	1.06944

Confusion matrix

	Predicted as Extreme	Predicted as Major	Predicted as Minor	Predicted as Moderate
Labeled as Extreme	0	0	223	0
Labeled as Major	0	0	647	0
Labeled as Minor	0	0	2623	0
Labeled as Moderate	0	0	929	0

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Precision Recall Curve

<< Go back

Summary of 2_DecisionTree

<< Go back

Decision Tree

• n_jobs: -1
• criterion: gini
• max_depth: 3
• num_class: 4
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

12.6 seconds

Metric details

	Extreme	Major	Minor	Moderate	accuracy	macro avg	weighted avg	logloss
precision	0.624113	0.54522	0.851328	0.599572	0.756671	0.655058	0.742191	0.591085
recall	0.789238	0.652241	0.940907	0.301399	0.756671	0.670946	0.756671	0.591085
f1-score	0.69703	0.593948	0.893879	0.401146	0.756671	0.646501	0.736552	0.591085
support	223	647	2623	929	0.756671	4422	4422	0.591085

Confusion matrix

	Predicted as Extreme	Predicted as Major	Predicted as Minor	Predicted as Moderate
Labeled as Extreme	176	43	4	0
Labeled as Major	88	422	53	84
Labeled as Minor	2	50	2468	103
Labeled as Moderate	16	259	374	280

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Precision Recall Curve

SHAP Importance

SHAP Dependence plots

Dependence Extreme (Fold 1)

Dependence Major (Fold 1)

Dependence Minor (Fold 1)

Dependence Moderate (Fold 1)

SHAP Decision plots

Worst decisions for selected sample 1 (Fold 1)

Worst decisions for selected sample 2 (Fold 1)

Worst decisions for selected sample 3 (Fold 1)

Worst decisions for selected sample 4 (Fold 1)

Best decisions for selected sample 1 (Fold 1)

Best decisions for selected sample 2 (Fold 1)

Best decisions for selected sample 3 (Fold 1)

Best decisions for selected sample 4 (Fold 1)

<< Go back

Summary of 3_Default_Xgboost

<< Go back

Extreme Gradient Boosting (Xgboost)

• n_jobs: -1
• objective: multi:softprob
• eta: 0.075
• max_depth: 6
• min_child_weight: 1
• subsample: 1.0
• colsample_bytree: 1.0
• eval_metric: mlogloss
• num_class: 4
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

15.7 seconds

Metric details

	Extreme	Major	Minor	Moderate	accuracy	macro avg	weighted avg	logloss
precision	0.683721	0.600634	0.89007	0.570248	0.777476	0.686168	0.770125	0.504764
recall	0.659193	0.585781	0.926039	0.519914	0.777476	0.672732	0.777476	0.504764
f1-score	0.671233	0.593114	0.907698	0.543919	0.777476	0.678991	0.77332	0.504764
support	223	647	2623	929	0.777476	4422	4422	0.504764

Confusion matrix

	Predicted as Extreme	Predicted as Major	Predicted as Minor	Predicted as Moderate
Labeled as Extreme	147	68	2	6
Labeled as Major	60	379	24	184
Labeled as Minor	2	18	2429	174
Labeled as Moderate	6	166	274	483

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Precision Recall Curve

SHAP Importance

SHAP Dependence plots

Dependence Extreme (Fold 1)

Dependence Major (Fold 1)

Dependence Minor (Fold 1)

Dependence Moderate (Fold 1)

SHAP Decision plots

Worst decisions for selected sample 1 (Fold 1)

Worst decisions for selected sample 2 (Fold 1)

Worst decisions for selected sample 3 (Fold 1)

Worst decisions for selected sample 4 (Fold 1)

Best decisions for selected sample 1 (Fold 1)

Best decisions for selected sample 2 (Fold 1)

Best decisions for selected sample 3 (Fold 1)

Best decisions for selected sample 4 (Fold 1)

<< Go back

Summary of 4_Default_NeuralNetwork

<< Go back

Neural Network

• n_jobs: -1
• dense_1_size: 32
• dense_2_size: 16
• learning_rate: 0.05
• num_class: 4
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

4.1 seconds

Metric details

	Extreme	Major	Minor	Moderate	accuracy	macro avg	weighted avg	logloss
precision	0.672986	0.577778	0.879392	0.552258	0.766169	0.670603	0.756126	0.565377
recall	0.636771	0.602782	0.925658	0.46071	0.766169	0.65648	0.766169	0.565377
f1-score	0.654378	0.590015	0.901932	0.502347	0.766169	0.662168	0.759863	0.565377
support	223	647	2623	929	0.766169	4422	4422	0.565377

Confusion matrix

	Predicted as Extreme	Predicted as Major	Predicted as Minor	Predicted as Moderate
Labeled as Extreme	142	74	3	4
Labeled as Major	60	390	27	170
Labeled as Minor	1	21	2428	173
Labeled as Moderate	8	190	303	428

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Precision Recall Curve

<< Go back

Summary of 5_Default_RandomForest

<< Go back

Random Forest

• n_jobs: -1
• criterion: gini
• max_features: 0.9
• min_samples_split: 30
• max_depth: 4
• eval_metric_name: logloss
• num_class: 4
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

logloss

Training time

14.0 seconds

Metric details

	Extreme	Major	Minor	Moderate	accuracy	macro avg	weighted avg	logloss
precision	0.643382	0.664368	0.87974	0.528541	0.768883	0.679008	0.762527	0.547393
recall	0.784753	0.446677	0.928708	0.538213	0.768883	0.674588	0.768883	0.547393
f1-score	0.707071	0.534196	0.903561	0.533333	0.768883	0.66954	0.761829	0.547393
support	223	647	2623	929	0.768883	4422	4422	0.547393

Confusion matrix

	Predicted as Extreme	Predicted as Major	Predicted as Minor	Predicted as Moderate
Labeled as Extreme	175	35	2	11
Labeled as Major	81	289	22	255
Labeled as Minor	2	5	2436	180
Labeled as Moderate	14	106	309	500

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Precision Recall Curve

SHAP Importance

SHAP Dependence plots

Dependence Extreme (Fold 1)

Dependence Major (Fold 1)

Dependence Minor (Fold 1)

Dependence Moderate (Fold 1)

SHAP Decision plots

Worst decisions for selected sample 1 (Fold 1)

Worst decisions for selected sample 2 (Fold 1)

Worst decisions for selected sample 3 (Fold 1)

Worst decisions for selected sample 4 (Fold 1)

Best decisions for selected sample 1 (Fold 1)

Best decisions for selected sample 2 (Fold 1)

Best decisions for selected sample 3 (Fold 1)

Best decisions for selected sample 4 (Fold 1)

<< Go back

Summary of Ensemble

<< Go back

Ensemble structure

Model	Weight
3_Default_Xgboost	4
4_Default_NeuralNetwork	1

Metric details

	Extreme	Major	Minor	Moderate	accuracy	macro avg	weighted avg	logloss
precision	0.691244	0.604615	0.889092	0.578624	0.780416	0.690894	0.772267	0.50403
recall	0.672646	0.607419	0.929089	0.506997	0.780416	0.679038	0.780416	0.50403
f1-score	0.681818	0.606014	0.90865	0.540448	0.780416	0.684232	0.775577	0.50403
support	223	647	2623	929	0.780416	4422	4422	0.50403

Confusion matrix

	Predicted as Extreme	Predicted as Major	Predicted as Minor	Predicted as Moderate
Labeled as Extreme	150	66	2	5
Labeled as Major	59	393	23	172
Labeled as Minor	2	18	2437	166
Labeled as Moderate	6	173	279	471

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Precision Recall Curve

<< Go back